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When gathering data for the design and evaluation of physical learning environments, 
there is a significant challenge in traversing, or translating, architect/designer language and 
stakeholders' verbal accounts of their expectations, preferences and experiences. In a series 
of studies used to provide data for both the evaluation of existing spaces and the design of 
future spaces, the authors utilized three focus group methods that incorporated elements of 
participatory design and visual play in activities. This paper describes these methods and 
the efficacy of each method in the context of physical learning environment evaluation and 
design. 


Introduction 

Over the past decade, shifts in higher education 
curriculum and pedagogical approaches, along with an 
increasingly diverse and technology-socialized generation 
of students, have prompted a re-examination of where, 
when, and how learning occurs in a university 
environment. As a result, campus design historically 
centered on the development of standardized, functional 
classroom, and lecture spaces has rapidly refocused on 
informal and technologically-enriched spaces. The 
discourse, once centered on functional efficiency, security, 
occupancy, and maintenance, has also refocused to be 
dominated by the effect of spaces on learning and the user 
experience. 

The ambitious and widespread nature of these 
redevelopments, and the substantial costs and risks 
involved, have given rise to a broadening of discussions in 
the university sector. In these discussions, the evidence 
base for developing new learning spaces, or lack thereof, 
has emerged as a key issue. The limitations of evaluation in 
relation to operational issues such as per capita floor space, 
cabling requirements for computers, service levels, or 
utilization and occupancy rates have also been brought into 
sharp relief by the new focus on user experience. Indeed, it 
has increasingly been argued that rigorous and multi¬ 
layered evaluations are needed in order to develop 
evidence-based models and knowledge for the design of 
learning environments (Johnson, 2005; Johnson & Lomas, 
2005; Mirijamdotter, Somerville, & Holst, 2006). 

There has also been an expansion in the use of novel 
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methods of data collection for the purposes of learning 
mixed-methods and more qualitative or interactive 
approaches to evaluation. These approaches span from 
participant and non-participant observation to focus 
groups and the use of informal stakeholder groups during 
the design phase. Individual studies cite the use of a range 
of observational activities, including photography, to 
document the way the space is used (Hunley & Schaller, 
2006). Personal reporting, such as the use of user- 
experience journals and blogs, has also gained in 
popularity as a means of data collection. In addition to 
these methods, focus groups appear to have gained 
significant traction as a preferred approach to engaging 
small groups of staff and/or students in both pre-design 
and post-occupancy evaluation (Learning Landscape 
Project Team, 2008; Lee & Tan, 2011; Woolner et al, 2007; 
2009). 

Over the past five years, the authors have been working 
in the design and evaluation of learning spaces at an 
Australian metropolitan university. Over this period, 
iterations of studies have been conducted into the needs 
and experiences of academic staff and students at both pre¬ 
design and post-occupancy stages. The intention of these 
studies was to gain a better understanding the efficacy of 
designs in relation to user experience, and to utilize post¬ 
occupancy evaluations in the development of new design 
specifications. 

Both quantitative and qualitative methods have been 
used in this process. This paper focuses on the focus group 
methods used, and in particular, the use of activities to 
supplement or supplant focus-group discussions. Three 
types of focus group activity were trialed, each with a 
particular data outcome in mind. As part of this process, 
the activities and their usefulness in gaining insights into 
the needs and perspectives of users were themselves 
evaluated. This paper describes the rationale for the 
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selection of those methods, the context and details of their 
implementation, and their efficacy in gathering detailed 
and useful information around the user experience that 
could then be fed forward to the next stage of campus 
design developments. 

Focus group methods in the literature 

Focus groups are a widely utilized research method and 
are often used as part of a mixed-methods approach. Focus 
groups, as a form of group interview, are commonly used 
to follow up after the analysis of survey data (Stewart, 
Shamdasani, & Rook, 2007) to allow researchers to explore 
emergent themes in greater depth. As Stewart et al point 
out, "one of the most appealing features of focus groups is 
their robust versatility for shedding light on almost any 
topic or issue" (p. 42). Focus groups are also commonly 
used for evaluative research or, less formally, as a means of 
gathering general stakeholder feedback on organizational 
activities. Stewart et al argue that focus group approaches 
can be useful in both exploratory and confirmation phases 
of evaluation. In other words, both as a mechanism for 
identifying stakeholder needs and as part of a review of 
outcomes. 

While focus groups have a number of advantages as a 
research method, including their ability to provide rich 
data very quickly (Silverman, 2001), they also have 
limitations, including the tendency for dominant voices to 
overtake the discussion, thereby privileging particular 
cohorts while excluding others (Cohen & Manion, 1997). 
Woolner et al (2009) also argue that, when used alone, these 
are severe limitations of verbal interview techniques. They 
further suggest that visual mediation activities are a 
valuable alternative for, or addition to, verbal or written 
data collection techniques. In particular, they argue that 
visual activities have significant advantages in situations 
where the participants are likely to be diverse in age, 
cultural background or linguistic ability (Woolner, et al., 
2010). Such activities can take on a range of forms, but 
typically involve "enhancing the traditional interview 
through using visual items, such as photographs, pictures 
or diagrams to mediate interviews and discussions" 
(Woolner, et al., 2009). The aim of this process is to produce 
a more interactive and formative process than that possible 
by verbal interview alone. 

The use of visual and interactive methods during the 
needs analysis stage of evaluation is also reflected in the 
literature around participatory design. Participatory design 
is broadly centered on the notion of collaborations between 
designers, researchers and user-participants (Blomberg & 
Henderson, 1990; Sanders, Brandt, & Binder, 2010; Zaphiris, 
Laghos, & Zacharia, 2009). The advantage of participatory 
design processes over user-centered design (in which the 


resear cher/designer defines the development process and 
interprets feedback from users) is that, depending upon the 
degree of interactivity, participants are able to directly 
engage in the creative processes of designing and/or 
problem solving. Sanders (2002) argues that the significant 
advantage of participatory methods is that tacit knowledge 
can be uncovered in a way not possible with verbal 
methods (p. 4) and that: "The new rules call for new tools. 
People want to express themselves and to participate 
directly and proactively in the design development 
process" (p.2). 

The three focus group methods described in this paper 
were intended to contribute to the exploratory and/or 
review stages of evaluation, and to focus participants on 
their experiences and preferences. Two of the approaches 
focused on an evaluation of user experience and had 
previously been used in primary school campus 
evaluations. The researchers devised the third approach as 
part of a needs analysis for the design of new learning 
environments. In the next section, the context of the studies 
is briefly outlined. This is followed by a summary 
description of each method, its use in context, and our 
findings regarding the challenges and benefits in relation to 
the intended purpose. 

Context of the Studies 

At Swinburne University of Technology, major campus 
developments have been the subject of evaluation at both 
pre-design and post-occupancy stages since 2007. Campus 
developments over that period have included a project hub, 
a student services building, and a nine-story teaching and 
research building dedicated to technology programs. A 
second multi-story teaching and research building is 
currently underway. The authors have undertaken research 
with faculty, administrative staff and students during the 
pre-design, building and post-occupancy stages of each of 
these building projects. Both quantitative and qualitative 
methods have been utilized together and separately, as 
appropriate to the research objectives. 

The approaches described in this paper arose out of our 
experiences of carrying out a pre-design needs analysis. 
While standard focus groups, using semi-structured 
interview formats, had proven useful in gathering data 
regarding the overall experience of stakeholders, and some 
insight into their needs for particular spaces, we found that 
there were limitations in the data which appeared inherent 
to the largely interview-based formats we were using. 
Specifically, that: 

• They were inadequate for translation between 
design and experiential languages 

• They yielded little useful data for a deeper 
understanding of design requirements 
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• There was a tendency for 'group think' or 
dominant members to overwhelm the data 

While the last of these issues, uneven representation of 
individuals in the data, is well documented in the literature 
as a limitation of focus groups, the relationship between 
design evaluation and focus groups has not been 
thoroughly explored. The core challenge we found in 
developing meaningful evaluations in the context of 
physical learning environments was that design was 
mediated by participants' capacity to think and talk in 
design terms and to relate daily experience with campus 
design issues. This led to a disconnect between the 
participants' experiences and involvement, and the actual 
design process. 

As a result, over subsequent iterations of evaluation we 
trialed three focus group activities with the intention of 
identifying mechanisms for more meaningful engagement 
with user-participants. 

The studies 

The three methods were trialed as part of the evaluation 
of two buildings. The first of these was the Project Hub, a 
student-focused learning environment located at the main 
Melbourne campus of the university. The Hub is a learning 
environment designed for use by final year students 
undertaking their capstone projects. Students work in this 
space without supervision; faculty wishing to use the space 
for tutorials must be invited by their students. The Hub is 
designed in several zones, each with a specific type of 
activity in mind. These are: open spaces with group tables, 
meeting rooms, a computer area, and informal social areas. 

Along with a survey covering student perspectives on 
the design, fit-out and relevance of the Hub design to their 
learning activities and needs, focus groups were held to 
gain further insights into the way that the spaces were used 
and student perceptions of the design of the zones. 
Participants were final year students drawn from across the 
disciplines (n-22) and aged 21-27. Two focus group 
methods were used: diamond ranking and visual mapping. 

Method 1: Diamond ranking 


The activity involves providing the participants with 
nine pictures of areas within a learning environment. The 
group or groups are given time to discuss preferences for 
particular areas and for what reason. Then, through group 
consensus, a diamond shaped ranking is produced 
showing the most favored areas at the top, to the least 
favored areas at the bottom. The groups are then asked to 
explain why they decided upon the arrangement they 
chose. Further discussion is used to explore both the areas 
of consensus and any disagreements within and across 
groups. 

This activity provides a prompt for conversation and 
gives a greater opportunity for all participants to influence 
discussions. Woolner et al. argue that using photographs 
for data collection prompts participants to provide 
commentary and perspectives that would not otherwise be 
given, and also mediates understanding more effectively 
than purely verbal responses. With sufficient numbers, 
diamond ranking can also be used to generate quantitative 
data by placing values on each image (James & Thomas, 
2008). 

We followed the prescribed procedure for diamond 
ranking, and provided groups of 4-5 participants with nine 
laminated photographs of the Hub. The outcomes from 
Groups 1 and 2 are shown as Figure 1, below. 


Group 1 



Group 2 



2 8 5 



Figure 1. 1. Computer area. 2. Creative area. 3. Exhibition area. 4. Booths. 
5. Social area. 6. Social learning. 7. Meeting rooms. 8. Laptop area. 9. 
Board rooms. 


The first method utilized a visual activity for gaining 
feedback regarding design preferences for a social learning 
environment. Originally developed by a British team 
(Woolner, Clark, & Ulrike, 2008; Woolner, Hall, Wall, & 
Dennison, 2007) carrying out school consultations for a 
government building project, the process falls within the 
"photo elicitation" range of methods. Based on a thinking 
skills activity, it is used in lieu of a simple question/answer 
procedure. 


The results of the rankings provided data regarding 
which areas students preferred in the Hub and, with the 
two groups independently choosing similar arrangements, 
affirmed the more and less favoured areas. Following 
completion of the ranking procedure, the groups were 
asked to explain their decisions, and to explain where 
individuals had differed in their preferences. As well as 
general discussion regarding the most and least preferred 
areas, deeper data were found when the groups described 
why they had chosen a particular hierarchy of preference 
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and where differences of opinion had arisen. Individual 
participants articulated preferences for quiet or social areas, 
types of aesthetic or responses to furniture, particular 
technologies and their varying usefulness for specific 
activities. Areas of agreement and disagreement equally 
prompted active discussion and detailed explanations. 

The diamond ranking gave participants an opportunity to 
argue for parts of the space, driving discussion around 
preferences that might have been less balanced without the 
visual mechanism. From our experience, we felt that the 
process did enable the kind of visual mediation and 
enhancement of active engagement described by Woolner 
et al (2009). In particular, the use of images allowed 
participants to first discuss their experience with one 
another without researcher questioning, and then to drive 
the following discussion. The process of ranking visuals 
also gave structure to the discussion and allowed 


they visit the space, marking locations they spend time in 
and making comments where appropriate. After 
completing their maps, participants are asked to explain the 
route they have drawn. Mapping in this way not only 
provides a point of discussion but also visual evidence of 
common trends and individual differences. 

For our purposes, the visual mapping tool was slightly 
modified. Again, we used the activity with groups of four 
to six students. Each participant group was provided with a 
different-colored pen and a transparent sheet to draw on, 
which itself was placed on top of a copy of the Hub map. 
By having each participant draw on the transparent sheets, 
it was possible to combine each by layering them on top of 
one another. This provided a visual means of identifying 
similarities and differences between each participant's use 
of the space. After completing their maps, participants were 
asked to explain the route they had drawn. 



Figure 2. Examples of visual maps completed by focus group participants. 


participants time to work through their ideas and 
elaborate on them. As researchers, we were left with a 
visual and written record of group preferences and their 
genesis. This, when combined with researcher notes, 
helped to validate our interpretations of the discussion, and 
gave depth to the analysis. 

Method 2: Visual mapping 

The same British team used visual mapping activities to 
elicit data around patterns of preference and usage. While 
diamond ranking helps to generate group decisions, visual 
mapping allows participants to provide data regarding 
their individual experiences. Each participant is provided 
with a pen and a map of the space. They are then asked to 
draw on the map a typical route that they would take when 


The findings from this activity were similar to, and 
confirmed the general hierarchy of preferences in, the data 
generated by the groups completing the diamond ranking 
activity. The findings also provided some new insights into 
the way that students selected spaces in a particular order, 
and why. Specifically, in this activity students were more 
focused on individual choices and what they did on entry 
to the space than a discussion of the design characteristics 
of each space. In other words, the discussion tended toward 
reflective descriptions of their perceptions of the 
environment as a social space, rather than aesthetic 
preferences. 

Nonetheless, similar to our experience of the ranking 
procedure, we found that the activity prompted a great 
deal of discussion and debate, with similarities of choices 
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becoming clear very quickly. The process also prompted a 
great deal of discussion about the limitations of the space in 
peak hours, when preferred spaces were not available, and 
the degree of social engagement in the space. This flagged a 
pattern of use that was opportunistic (looking for friends) 
and indicated that the Hub had become a focus of campus 
visits, something that was not evident in the diamond 
ranking groups. 

However, despite providing more insight into the 
broader social aspects of space use, in terms of 
understanding design needs, this method resulted in more 
general findings and was less effective than the diamond 
ranking. As we were evaluating the use of a relatively small 
and defined learning environment, the findings around 
navigation were also limited. The method is possibly more 
effective in a more complex and larger environments, 
where traffic patterns are less able to be observed and 
recounted through other methods. 

Method 3: Evaluation by design 

The third method we trialed was for the post-occupancy 
evaluation of a new teaching/research building at the same 
campus. In this case, the focus groups were convened to 
identify how the new group-oriented and technology- 
enabled classrooms were meeting staff and student needs. 
Because the rooms in question had been piloted with 
previously un-tested layouts and technology, we were 
particularly concerned with their usability. In particular, 
we hoped to gain sufficient understanding of the 


experience of pilot rooms to improve the designs for future 
use and/or retrofitting of the current building. 

Again, we used a survey to elicit general feedback and a 
focus group to gain deeper insights into the experience of 
users. Focus group participants (n=9) were faculty who had 
used the new rooms during the first semester of their 
implementation. Participants were aged 32-48 and came 
humanities, business, information technology and 
engineering. 

While all of the methods are potentially useful for 
gaining insights into how learning environments are 
experienced, and therefore how new spaces might be 
designed, this method was more explicitly forward- 
looking. We described the process to participants as one 
that was intended to draw on current experiences to 
identify effective characteristics for classroom design. These 
findings would then be used to design possible changes to 
the existing rooms, and for new developments. 

For this purpose, and based on concepts drawn from 
participatory design, we developed a basic design activity 
to facilitate discussion around classroom fit-out. The 
activity involved the use of abstract objects as tools to work 
through the design problems of creating the "perfect" 
teaching environment. As with the first two methods, the 
intention of the activity was to assist participants to 
articulate their experiences and thoughts, prompted by the 
questions raised by the activity. We used a collection of 
everyday items such as pins, string, ice-cream sticks. Lego, 
beads, play-dough, marker pens, blue-tack and sticky note 
pads. The scale, color, and number of objects were selected 
with the intention of providing sufficient and flexible items 
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that would facilitate creative solutions, promote positive 
play and not confuse or overwhelm participants. 

Participants were provided with a collection of these 
items in groups of two to three. An art board and a 
selection of play objects were provided to each group. We 
provided participants with verbal instructions for the 
activity at each stage. The first instruction was for 
participants to imagine, if they were given total control, 
how they would design a teaching room. They were asked 
to consider the details of such a space, including the kinds 
of general equipment and furniture they would use, and 
the placement of each. We then gave participants 15 
minutes to create their first design. The second instruction 
was to consider the use of technology, and 10 minutes were 
given for this. The final part of the activity was a whole-of- 
group discussion regarding the choices made, justifications 
and comparison of similarities and differences between the 
designs. 

The activity prompted a great deal of creative discussion, 
argument and laughter as participants interpreted the 
objects and attempted to work through the problems of 
designing a complete space. The level of interaction was 
very high as was the level of participant-driven discussion 
and debate following the activity. We also saw an increased 
level of creative solutions to problem experiences between 
stakeholders than that gained in the standard focus group 
interviews. While the debate was perhaps more wide- 
ranging than that found in interview-type processes, we 
were able to intervene with specific questions and 
clarifications that provided substantial data on the 
affordances and barriers to teaching and learning in those 
spaces, and the variation in use from group to group. In 
particular, we gathered data around the common patterns 
of one-to-one, one-to-group and group-to-group 
communications affecting furniture and technology use, 
particularly the use of multiple screens and their control. 

A secondary, but useful, outcome of this process was a 
formative element. Participants reported gaining a greater 
understanding of the complexity of designing a space. 
Participants also reported that they appreciated the new 
spaces more following attempts to design their preferred 
environment, as the activity had given them insights into 
the conflicts between individual needs and the 
accommodations that needed to be made in designing for 
multiple uses. 

That is not to say the activity was without challenges. 
Some participants appeared to be intimidated by the 
openness of the activity and to be confused by the use of 
abstract items, having expected a question-answer format. 
Yet, within 10 minutes participants were actively engaged 
in imagining what the objects could represent and working 
through the problems. Participants also reported that they 
would have liked longer than 30 minutes to complete the 


activity, despite the scheduling challenges involved. The 
feeling of time constraint may also explain why participants 
tended to use very basic forms of representation. Most 
groups used mostly blocks, marker pens, and pipe cleaners, 
making use of annotation to explain what was being 
represented (see Figure 4). 



Figure 4. Examples of environments designed by focus group participants. 


Discussion and conclusions 

Focus groups are time consuming to carry out and 
analyze. Qualitative data can also suffer from being skewed 
by small and self-selecting respondents. In our case, the 
focus groups also suffered from low face-validity in 
comparison to the companion surveys. Nonetheless, focus 
groups can be used to achieve in-depth insight, and to 
allow for exploratory discussions with participants. We had 
previously carried out focus groups using semi-structured 
interview formats, but these had also suffered from a lack 
of depth and, during analysis, required a great deal of 
researcher interpretation for design purposes. We also 
found that the most important aspect of the data, why users 
found some design characteristics useful, was difficult to 
interpret from verbal responses alone. As a consequence, 
transfer of evaluation findings from one context to another 
remained problematic. 

As noted by Woolner et al (2009), visual methods are 
especially effective where verbal or textual information is 
difficult, particularly when comparing across groups with 
varying verbal skill (e.g. children and adults). We would 
further argue that both visual and design-based methods 
are equally useful where there is a need to mediate between 
verbal, visual, and spatial languages, and particularly 
where participants are not familiar with one or more of 
those languages. We found that the visual and interactive 
nature of the activities, when used in a focus group context, 
was an effective mechanism for engaging participants in 
more complex discussions and facilitated equitable and 
active contributions from individuals. Specifically, that the 
visual cues provided a common point of entry into 
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discussion. Further, we found that the visual and design 
processes used helped to mediate participant capacity to 
use design language, and the researchers' capacity to 
interpret perspectives articulated by participants. 

While we found that diamond ranking and visual 
mapping provided very similar data sets, they also had 
sufficient differences to provide distinct insights. The 
evaluative design activity took a generative approach but 
equally provided rich insights into participant experience 
and perspectives. These went beyond a measurement of 
satisfaction with particular learning environments, giving 
us a greater understanding of why particular designs were 
preferred and how they facilitated the work carried out by 
students and staff. When used in combination with survey 
material, the data supported the transfer of design 
characteristics to new contexts. 
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